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Agenda 


> CIVR Overview and Demo (15 minutes) 
> Alignment with Digital Service Standards (30 minutes) 


e Questions and Discussion (30 minutes) 
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CIVR Overview 


Background 

«+ To support MTO’s Digital Transformation Strategy, as well as the Contact Centre Modernization Initiative (CCMI). 

+ Replace the current touch-tone D&V Interactive Voice Response (IVR) application with a Conversational IVR application making it 
easier for customers to reach an agent, increase self-service and reduce the number of calls. 

«+ Over 1.7M calls are received by the D&V IVR application annually; 15% receive busy signals; only 12% can be self-serviced 


Objective(s) 

- Expand self-serve capabilities enabling a more efficient delivery of information to callers, and improved experience 

- Simpler routing of calls to agents, when this is the best option (e.g., complex enquiries, enquiries requiring access to case/client info) 
+ Create the foundation for future transition to Conversational IVR applications broadly in the OPS 


Approach 

e Research and analysis, to influence design (data, user research, journey mapping and current usage — types of calls, trends, human 
behaviour) 

» Leverage experience from the contact centre, learnings from D&V Chatbot, Lean Assessment, etc. 

< Leveraging new technology, and technology partners (e.g., Cloud/Amazon Connects) 
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Problem Statements 


How a call moves through the IVR system 


—) Menu —> 


‘ <> 
@ —) t- —) FAQs —_—) 
PD comer — GOD 
Customer IVR system If necessary, 
calls you call is routed 


1) Difficult to navigate the current system and self-serve aaam 


Individuals or businesses calling the ServiceOntario Interactive Voice Response (IVR) for the purposes of 
getting information regarding Driver & Vehicle (D&V) services are not able to effectively self-serve. 


Users must navigate multiple layers of the ServiceOntario IVR and the specific D&V branch of the IVR to 
receive a response to their enquiry, which may cause frustration with the length of time spent in the IVR 
or reaching the MMR (multi-media routing) to speak to an agent. 


Data shows that callers listen to most of the message for a topic they have selected in the IVR before they 
press the option to zero out (most of message = 75% of message). With CIVR, users will only hear 
messages that are pertinent to the specific question they ask. 


2) Out of space for new programs 


ServiceOntario's ability to effectively support any additional MTO D&V programs (including Covid-related 
programs) is limited due to the IVR reaching capacity and complexity in its current state (5X5 menu 
structure). MTO introduces new D&V initiatives on a regular basis, making it challenging to add these new 
programs in the IVR as others would need to be removed. With the CIVR, programs/new information can 
be added more efficiently, allowing the channel to be more agile. 
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Caller Experience Transformation with CIVR 


Replace the current touch-tone 
D&V IVR, with a conversational 
IVR, so that callers: 


- will not need to navigate 
multiple layers of menu 
options to speak to an agent 


» will have more opportunities 
to self-service, increasing 
customer satisfaction and 
reducing the number of calls 
that go to an agent. 


* access to more information 
as CIVR is not limited to the 
amount of information like 
today’s IVR 
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Conversational 
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Solution in1 
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Alignment with Digital Service Standards 


The next slides provide a summary note for each of the DSS principles. 


1) 
2) 
3) 
4) 
5) 
6) 
7) 
8) 
9) 


10) 
11) 


Understand users and their needs 
Establish the right team 

Be consistent 

Design the service from start to finish 
Make it accessible and inclusive 

Be agile and user-centred 

Use open standards and common platforms 
Embed privacy and security by design 
Support those who need it 

Measure performance 

Be a good data steward 


Note: for further reference see the after-Discovery document and survey presentation. 


7 Conversational Interactive Voice Response (CIVR) Onta rio © 


MPBSD 23-071 Record 12 CIVR afterDiscovery Final 


1. Understand users and their needs 


Understanding the needs of people who use the service is critical to building 
something that works for them. 


Moving to a Conversational IVR system, from an Interactive Voice Response system, 
is an opportunity consider humanizing the call experience. Doing so would enable a 
less robotic feeling, for callers, and continuing to provide high value informative 
responses. 


Because there are very few leading examples of successful CIVR systems, our 
opportunity has been to learn from our users some core considerations as we work 


through the discovery phase of the project. 


User research included a survey and listening to call recordings. 
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1. Understand users and their needs (continued) = 
Survey Executive Summary 6 . = 
Online Survey Goals: dE 


e Discover the expected ‘tone of voice’ of a Voice entity in an CIVR system 
e Confirm the content design, length and messaging of the existing IVR responses is still appropriate 
e Confirm how callers expect a Voice Entity to respond to their phone enquiry 


The following high level, persona and/or experiential insights have been identified: 


1. Information provided (via phone agent) should be delivered in a friendly tone, with the opportunity to follow-up with 
additional questions 


2. A single response using existing the (existing) IVR/Web content was preferred over creating a new multi-prompt responses 


3. There was no clear preference in the personality of a phone support voice entity; the least preferred option was one that 
sounded direct and formal 


4. Multiple write-in responses contrasted that call messages were generally too long, but could be considered a reference to 
the IVR experience in total. 
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1. Understand users and their needs (continued) 


Call Recordings Executive Summary 


User research included analyzed to >800 call recordings. Call recordings indicated: 
1. 28% of calls required look-ups by agents and cannot be self-served. 


2. Top questions (for zero out calls) 


Delivery issues 
Online transactions (e.g., payment issues, errors, didn’t receive confirmation) 


Driver and License history/abstract 
Appointment booking 
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1. Understand users and their needs (continued) 


Call Recordings: examples of callers and questions 


People with disabilities 


How do | get an 
accessible parking 
permit (APP)? 


My APP expired but | 
did not get a renewal 


How do | get a photo 
card? 


How do | renew my 
photo card? 


New drivers, >80 years 
old 


| want to apply for a 
driver’s licence 


Where do | go to get a 
driver’s licence? 


I’m over 80 now so 
what is the procedure 
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Licenced drivers 


| want to renew my 
driver’s licence 


| want pay fines 
| lost my licence 


| received a renewal 
notice. What do | do? 


How do | get my 
suspended licence re- 
instated? 


People in D&V process 
waiting for fulfillment 


When will | receive my 
vehicle sticker? 


My licence has not 
arrived in the mail, 
when will it come? 


There is a problem 
with my payment | 
need to fix 


Vehicle owners 


| lost my licence plate 
and need to replace 


| need to replace my 
ownership 


| need to replace my 
permit 


| need to replace my 
sticker 


| want to transfer my 
car to a family member 


New to Ontario/ change 
of information/ 
travelling 


How do | change my 
name or address for 
my licence/vehicle/ 
photo card? 


I’m visiting Ontario: 
can | drive here? 
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2. Establish the right team 


CIVR Team (core) 


Project Leadership 

Maintains overall accountability for project 
outcomes and eae AUGlESs manages executive 
client relationships and helps triage escalated 
risk items 


Project Core Team 

Responsible for day-to-day planning and 
execution of the project, collaborating with 
stakeholders and escalating risks/barriers 


Subject Matter Specialist 
Responsible for providing specialty knowledge of 
CCMB business and technology transformation 
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Executive Sponsor: 
Olga Bakonyi, Head, Contact Centre Modernization Branch 


Product Owner: 
Scott Gow, Senior Manager, Business Solutions, CCMB 


Product Manager: 
Tim Trudeau, Senior I&IT Project Manager, CCMB 


Vergel Evans - User Experience (UX) Designer from MTO Digital Garage 
Kyle Kwan — Content Writer, User Researcher (Business Liaison) 

Lyn Lim — Content Writer, User Researcher (Business Liaison) 
Matthew Hayvren — QA Specialist 

Annie Perez — Technical Lead 

Carolyn Luney - Service Designer 

Charith Senaratne - |&IT Lead Technology Specialist 


Ontario © 


MPBSD 23-071 Record 12 CIVR afterDiscovery Final 


2. Establish the right team (continued) 


Extended team of subject matter experts & stakeholders 


MTO SMEs 

Keith Madill — Sr. Policy Advisor, Driver Program Development 

Arthur Legardo — Sr. Policy Advisor, Driver Program Development 

Bob Rajhans — Policy Team Lead, Vehicle Program Development 

Melody McClelland - Manager, Program Standards, Licensing, Permits and Support Office 
Jennifer Elliott - Manager, Licensing, Permits and Support Office 

Julia Scorcia - Team Leader, Contract & Oversight Management Office 

Maryanne Cirillo - Business Advisor, Contract & Oversight Management Office 

Allison Partington, SME Digital Service Playbook from MTO Digital Garage 


MGCS 

Privacy and Records Management 
Cyber Security 

Enterprise Architecture 


MGCS, ServiceOntario SMEs 

Rover Okumura, Operations Manager, Customer Care 

Praveen Senthinathan, Ashleigh Cocuzzi and Derek Kinley - MGCS Communications 
Alessia Vaughan and Chuck Jane — Contact Centre Services 

Sarah Loveridge and Trish Lewis — Change Management Implementation 

Carine Chartier — Reporting & Information Management 

Stephanie Guillen — Business Training Unit 
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3. Be consistent 


When the public interacts with the Ontario government, their experience should feel 
cohesive, positive and consistent. 


- Driver and vehicle is an eco-system. Different services are 
interdependent, for example from user research: Because the DriveTest 
Contact Centre line is busy, drivers are calling ServiceOntario. Problems 
from one area are transported to another area with people looking for 
help. 


e We are working with Ontario.ca and the chatbot (web channel) for 
increased consistency and support for customers. 


> CIVR will direct callers to the Ontario.ca website, although we do not 
anticipate a traffic surge to Ontario.ca. This is not changing from the 
current state. 


- The Contact Us information on the Ontario.ca webpages will not change. 


KKS 
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4. Design the service from start to finish x5 
x 


Understand what users are trying to do and how it fits with the broader 
context of their life. The user journey in the contact centre process would 
benefit from the centre of the journey noted below as this is where gaining 
access to the answer(s) to their question(s) may be resolved. 


Conversational IVR 
Project Scope 


(---4-—=> 


2. Research (ma l 
1. Problem to may l 
include website, 
solve or : : 3. Call the number 
social media, 
information to and enter the IVR j; 
3 word of mouth, selections 
find I 
referral, etc) 


8. Problem 
resolved or 


4. Listen to IVR 
and make 


I 5. Press zero and/ 6. Speak to agent 
or waitin queue once an agent is 
for an agent available 


information found 
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5. Make it accessible and inclusive 


Make sure the service or product is accessible to all users regardless of their abilities, 
device, environment or quality of access. 


The team has consulted with the Ontario Accessibility Centre of Excellence for guidance regarding accessible practices for 
conversational IVR. The Web Content eeu Guidelines (WCAG) do not specifically relate to CIVR. The consultation 
provided some recommendations associated to the on the overarching principles: 


Perceivable: information must be presented to a user in ways they can perceive 
Operable: users must be able to use the interface and it can’t require an interaction that the user can’t perform 
Understandable: users must be able to understand the information provided and how to use the interface 


We will work with the content owner on the wording used and ensure it is in both official languages. User testing will provide 
additional insights into accessibility and usability before launch. 


Some user groups will benefit from the conversational IVR technology such as: 
people with visual impairment who don’t use or rely on visual or text-based information 
people with mobility or dexterity impairments who find buttons difficult to press 


Note: the capability of a caller to connect to an agent or use TTY will continue to be available options 
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6. Be agile and user-centred 


Build and test one feature of the service or product at a time and work towards 
continuous improvement. 
Smaller packages of work will be identified to demonstrate the functionality through prototyping 


Collaboration will continue using the tools identified e.g. Miro, MS Teams, SharePoint. Regular touchpoints include 
2-3 stand-ups per week 


Workshops for content design include SME’s from with Policy, Communications, Technology, User Experience, 
Quality Assurance and Business/Program 


A core discovery from the online survey has been the importance of keeping messages as close to existing 

content as possible. From the comments in the survey, respondents are looking for concise information and do not 
want additional steps, or segments in the audio before receiving all the required information to feel their 
information request has been answered 


Listened to >800 calls into ServiceOntario indicated that agents asked what type of licence a caller wanted to know 
about 


Team will be able to adjust to needed improvements 
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7. Use open standards and common platforms 


Use open standards, open-source software and common government platforms where 
available. 


Voice-activated IVR applications are being hosted on ECCS today (e.g. 511, MOF), and this is the next-generation 
technology 


This MVP will be the first of its kind in the Ontario government and will provide learnings that can be leveraged for 
future CIVR implementations within the OPS 


Research has included jurisdictional scans to gain a better understanding of how to solve problems identified 
The CIVR administration interface will leverage Single Sign-On (SSO) for the administrative users 
The CIVR solution (used by the public or public facing) will integrate with the existing on-prem ECCS contact centre 


A variety types phones will be used to call into the CIVR 


Note: the general public will only be able to interact with this service by placing calls 
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8. Embed privacy and security by design 


Put appropriate legal, privacy and security measures in place so that users feel confident 

that their personal information will be kept secure, and their privacy will be respected. 
CIVR will collect caller’s utterances (what they are speaking to the CIVR. For example “How do I renew my driver’s licence”, 
“Yes, that’s what | want”. Additionally, standard call metrics will be collected such as caller’s phone number, phone number 


that the caller dialed to reach the CIVR (this may be a conversion phone number associated with a toll free number) and 
call time duration. 


Call recordings and transcripts are stored in S3 buckets (repository) - these buckets will be managed by administrators 


We are engaged with the MGCS Privacy Office to determine if an assessment and or any changes are recommended. 


A TRA is scheduled for January 2022 
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9. Support those who need it EEI 


Put tools in place across all channels to support people who cannot use digital services or 
products on their own. 


Based on research, examples of groups who may experience accessibility barriers include: 
e Individuals with hearing impairment 

e — Individuals with cognitive disabilities 

< Individuals with speech and language impairments 


These groups will be considered during the design of the solution, and accessibility accommodation will be incorporated in the call flow design. 
Mitigation strategies may include: 


TTY (Teletype): callers who have their own TTY device can call into ECCS, when the IVR receives TTY calls, they are transferred to another phone 
number and managed by the OPS Enterprise TTY service where an agent can communicate with the caller through text using the OPS TTY 
software installed on their desktop. 


Our ECCS standards is to always provide an out for the callers, therefore saying “Agent” or pressing O will be made available in the 
Conversational IVR design. “Timeouts” can also be used to route callers that don’t say or enter anything. The CCMB standard is never to “trap” 
the caller in the IVR whether it is a touch-tone, speech or conversational IVR, there is always an exit to the contact centre whether purposefully 
selected (via saying Agent or pressing 0) or unintentionally selected (by timing out in a menu, repeating the menu content multiple times, or 
erroring out by saying/pressing something that was not expected). 


The appropriate updates to support models, knowledge records and administrative resource training is incorporated into the project 
deliverables to ensure a smooth transition to operational support and therefore maintain a high level of service for callers. 
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10. Measure performance 


Customer 
1 i : 
Satisfaction 
2 Speed to 
Need 


3 % Self Serve 


Early Agent 
Assistance 


Client satisfaction with answer 


Client satisfaction overall 


Time to first topic 


Time to MMR 


% hangups 


% zero out from top menu 


There is no target change because we don't have this measurement today. 
This would be measured by looking at callers who answered yes when asked within the conversation if their question was 
answered. 


Before we can set "target change" | think we would need a baseline? We can set a baseline by deploying a survey pre-cutover. 
Then running the same survey post cutover. 

For text based chats we said that the target for overall customer satisfaction (for both the bot portion of the chat and the agent 
portion of the chat) should be 80%. 


The baseline is the time it take the caller to get to the "topic" in today's IVR. 

In CIVR this would be measured by looking at time it takes caller to get to first topic. We can compare against baseline to 
measure improvement. 

The baseline is the time it take the caller to get to MMR from today's IVR. In CIVR this would be measured by looking at time it 
takes caller to get to MMR. 

We can compare against baseline to measure improvement. 

The baseline can be measured in today's IVR (i.e. number of callers that hanging /total callers). We know that most callers listen 
to most of the message before hanging up. 

In CIVR we'll want to measure the same thing (number of callers that hang up after listening to most of message) and ensure that 
it is not decreasing. Inspirationally it would increase, but success is maintaining. 


Note: Do we need to look at this after hours vs during business hours. 

The baseline can be measured in today's IVR (i.e. number of callers that press O from top menu/total calls). In CIVR we'll want to 
measure the same thing (number of callers that ask to speak to an agent/press 0 from "top menu). 

Note: If we are able to measure at other menus in CIVR we'll do that as well but will depend on design which won't be finalized 
until alpha. 
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11. Be a good data steward 


Follow the rules and best practices when you organize and manage data, including 
making data open by default. 


- ServiceOntario wait times and call volumes (contact centres) are published to the 


Ontario data catalogue: ServiceOntario wait times and call volumes (contact centres) - 
Datasets - Ontario Data Catalogue. This includes Driver and Vehicle Licensing (D&V) 
data. 
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QUESTIONS 
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What is a Conversational IVR? 


TODAY (Traditional IVR) 


Callers listen to scripts and use 
touch-tone to select, navigate 
options 


Callers navigate through many 
menus to find the information 
they are looking for, or they zero 
out to speak to an agent 


IVR scripts and menu options are 
updated when new programs are 
added or changed 
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TOMORROW 
(Conversational IVR) 
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| use natural language 
processing, artificial 
intelligence (Al), and 
machine learning to 
understand callers and 
provide them with the 
information they need. 


e 
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Ath Point from Survey - Callers expect a level of frustration with IVR 


“| like when the information I’m looking for is in 
one call and | don’t have to pick 10 other options 
to gather the information I'm looking for.“ 


“| often re-play audio prompts multiple times when I call 
Service Ontario. This is either because they're too long and | 
end up forgetting bits of information or because | want to be 
certain that I'm understanding the message correctly. | 
generally don't see this as a pain point (to replay messages 
multiple times). However, it does become a pain when the 
audio messages are too long - | end up feeling like I'm 


a f . wasting too much time. .“ 
Usually when | listen to these messages | will 


write down the information.“ 


“As much as possible, limit the number of times the 
“Not clear how to navigate calling options to get to the service | am looking user has to choose an option to continue. “ 

for. Agents are rushed to answer and the focus is on how fast they can 
serve you verses providing accurate information. Agents are equipped for 
basic questions, and questions that are more complicated are generally 
brushed off as the agents are stuck from trickling up and having a follow- 
up response system in place. I've been hung up on while in a call and 
cannot reach the same agent. Disappointed that once you open a 
call/case, you cannot continue to deal with the same person until the case 
is resolved - causes confusion. Provided with information that does not 
align with program area in person service.“ 


“Tell me where to find VIN Registration / Permit / Ownership 
/ green paper / certificate are what people call this 
document and will be less likely to understand the word 
permit. When people think of being pull over by law 
enforcement they - licence and registration please. .“ 
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MVP Evaluation 


1.Provide up-to-date, accessible information about MTO D&V programs and services, available to 
customers 24/7 

2. Instill customer confidence in MTO D&V programs and services 

3. No further requirement to repurpose / combine MTO D&V programs/services 


Successful ability to add additional D&V 


Address/Ensure Program Integrity programs/services to the Conversational IVR 
solution 


To improve customer experience and introduce 1.Provide up-to-date, accessible information about MTO D&V programs and services, available to 


efficiencies by allowing callers to get to the customers 24/7 
Improve Customer information more quickly (via IVR or agent) 2. Enable customers to obtain required information by self-serving within the Conversational IVR 
Experience/Introduce Efficiencies 3. Enable customers to reach agents in more efficient manner 

MTO has introduced a Digital Transformation j 1.Digital Transformation Strategy will enable MTO, in collaboration with MGCS, to deliver programs, 
E ER Strategy which includes the redesign and rebuild of services and products to Ontarians while following provincial guidelines for physical distancing and 

the D&V portion of the ServiceOntario IVR protecting vulnerable populations due to the current pandemic. 


2. To go beyond the current standard 5x5 touch-tone IVR design and to support more contact types 
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